A hypothetical reference decoder is valuable in compressed video systems because it serves to validate an encoded bitstream for compliance to a standard. In a coding standard such as H.264/AVC, there are numerous interoperability points due to the Scalable Video Coding features of the standard. The H.264/AVC standard has rules (also referred to as requirements, constraints, or operational specifications) defining an HRD. The HRD behavior is normative. Any AVC bitstream has to be compliant to the HRD built according to the rules. SVC (scalable video coding) is an extension (Annex G) to the AVC standard. An SVC bitstream typically has multiple interoperability points (also referred to as operation points), due at least in part to the fact that the bitstream is scalable. Such a bitstream may be scalable spatially, temporally, and in SNR, for example. Sub-bitstreams, corresponding to the scalable aspects, may be extracted from the bitstream. Previous HRDs do not have sufficient rules to allow them to validate bitstreams like those of the SVC in AVC.